1. Field of the Invention
The present invention relates to a device and program for producing a panoramic image by stitching multiple images together.
2. Description of the Related Art
Recent years have seen proposals for technologies that use computer processing to build an image-based wide-angle panoramic image by using multiple images of different angles captured by devices such as digital cameras.
Japanese Application Laid-Open Publication No. 2005-151563, for example, discloses such a method that lowers the resolution of high resolution images captured by a camera set to panoramic mode to create video frames. The motion between reference video frame and input video frame is estimated, and if the estimated value of the motion is greater than a specified predetermined value, a new high resolution image is captured and its video frame becomes the new reference video frame. Estimated motion data of the reference video frame is used as the initial value to compute the relative position of the high resolution image, then the high resolution images are stitched together to produce a mosaic panel (i.e. a panoramic image).
Japanese Application Laid-Open Publication No. 2004-247793 discloses such a method that produces a panoramic image by continually offsetting the subject area while tracking local feature points of the continuously shot images.
A challenge for existing methods that stitch newly captured high resolution images into the existing panoramic image (panoramic canvas) has been to produce a natural looking panoramic image in real time while compensating for the distortions between the images caused by changes in distance to the subject, angle, etc.
If camera is panned or tilted only in a single direction, horizontal or vertical, then the methods disclosed in the patent application documents above are sufficient for stitching images together by tracking successive local image feature points and compensating for the new images. However, when producing a 2-dimensional panoramic image by moving the camera both horizontally and vertically, or in a diagonal direction, or by changing the panning or tilting direction, new need arises to stitch the input image together with an image that has been captured more than one frame before the current input image. In this case, it is required to avoid producing a panoramic image with unnatural junctions caused by error accumulation.
The problem with existing panoramic image production methods that track image feature points is its requirement to perform highly accurate coordinate matching with the tracked image feature points that were previously detected.
Panoramic image production method stated above applies to single direction panning, therefore the reference frame and image feature points of the immediately previous frame is used, and older data can be discarded, thus reducing the memory requirement. On the other hand, when producing a 2-dimensional panoramic image, the problem of how to reduce the memory and processing power requirements while storing data that are older than from the immediately previous frame becomes important.